The present invention relates to software verification, and more specifically, to operating system aided code coverage.
Code coverage is a measure used in software testing, describing the degree to which source code of an application program has been exercised. Code coverage may quantify what portions of the application program were executed by a given test or series of tests. Collection of code coverage data is typically performed through directly instrumenting the application by adding additional code to monitor application parameters during execution. While running the instrumented application program (or at its termination) typically a coverage file is created. This approach also introduces overhead to the application execution due to the additional instrumentation code. Additional application-level support for a file system may also be required to store coverage information. For example, the application under test may be modified to write data gathered via instrumentation, injecting further changes into the application.
Instrumenting an application may not be effective or feasible for a number of scenarios, such as performing coverage at system level tests, in real-time applications, or where storage is inaccessible to the application for logging coverage information. System level tests may not support modification to add instrumentation, since the application is configured as it will be received by the customer (e.g., final release version) which makes modifying the application for instrumentation unacceptable. In a real-time application, the additional overhead associated with instrumentation may place too great of an additional burden on the system to maintain timing and memory allocation constraints for the application.
A less intrusive technique may employ sampling of hardware performance counters using special performance monitoring hardware to detect which part of the code was executed. Sampling hardware performance counters, however, is not always available, especially in embedded systems. Hardware performance counters add to the complexity and expense of processing system hardware, and may introduce new hardware failure modes should the performance monitoring hardware fail. Moreover, sampling of hardware performance counters introduces additional overhead and intervenes in normal execution of the program since the sampling is typically monitored by special kernel or user space application.